Wordpress SEO - On-Page SEO for your Wordpress Site (Webmaster Series) by Williams Dr. Andy

Author:Williams, Dr. Andy [Williams, Dr. Andy]
Language: eng
Format: epub
Publisher: http://ezseonews.com
Published: 2013-08-04T04:00:00+00:00

14.3.2. Non-Genesis Users

For non-Genesis users, you’ll need to see whether or not your template has an author bio box feature. If it does, you can use that. If it doesn’t, there are some plugins that add this functionality to any WordPress site. I cannot recommend any particular plugin as I don’t use one. You should be able to find some by searching the WordPress plugin directory (http://wordpress.org/plugins/).

15. Robots.txt File

The robots.txt file is a plain text file that contains various instructions for the search engines. It includes details on specific folders and files that the search engines should, or should not spider.

For example, you can tell the search engines to ignore all files within a specific folder on your server. A good example of a folder that search engines should keep out of is the WordPress plugin folder. The files there are not content pages on your site, so why would you want the search engines sniffing around in there? If you allow them to, they’ll index the PHP files in those folders, and so list them in the SERPS as content on your site. Not really what we want is it?

Using a robots.txt file is easy enough. Just create a text file in any text editor, and add in the commands. Once done, simply upload it to the root folder of your WordPress site.

On the WordPress website, they show a suggested example robots.txt file. You can find it here:

http://codex.wordpress.org/Search_Engine_Optimization_for_WordPress

However, since we are using Yoast’s WordPress plugin, a lot of what we need to do in terms of excluding search engine spiders, is done for us. In a post on the subject (http://yoast.com/example-robots-txt-wordpress/), Joost de Valk (the author of the WordPress SEO plugin), goes through the WordPress example and explains why most of it is not really necessary.

In Yoast’s article, he suggests a much simplified robots.txt file, and it’s similar to the one he currently uses on Yoast.com. Here it is:

User-Agent: *

Disallow: /wp-content/plugins/

I personally add a couple of extra lines to mine. The first one is to disallow spiders from going into the cgi-bin folder:

Disallow: /cgi-bin

The majority of site owners never have any files and sub-directories in this folder anyway, so it’s not critical for most. The other line I add to my robots.txt file is one that tells the spiders where my sitemap is located.

You can find the location of your sitemap by going into the XML Sitemap settings found in the SEO menu in the left sidebar.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.